System and method for generating expertise based search results

ABSTRACT

Embodiments of the present invention provide systems, methods and computer program products for generating search results comprising web documents with associated expert information. One embodiment of a method for generating such search results includes receiving one or more search queries, selecting one of the one or more search queries, determining one or more categories of web documents responsive to the selected search query and crawling a web graph of linked web documents to identify one or more web documents tagged as within the one or more categories responsive to the selected search query. The method further includes generating a result set of the one or more web documents identified as within the one or more categories responsive to the selected search query, ranking the result set and generating a list of ranked search results responsive to the selected search.

CLAIM OF PRIORITY

This application is a continuation of and claims priority to U.S. patentapplication Ser. No. 11/965,021, filed Dec. 27, 2007, which isincorporated herein by reference in its entirety.

COPYRIGHT NOTICE

A portion of the disclosure of this patent document contains materialwhich is subject to copyright protection. The copyright owner has noobjection to the facsimile reproduction by anyone of the patent documentor the patent disclosure, as it appears in the Patent and TrademarkOffice patent files or records, but otherwise reserves all copyrightrights whatsoever.

FIELD OF THE INVENTION

The invention disclosed herein relates generally to generating expertisebased search results. More specifically, the present invention isdirected towards systems and methods for receiving a search query andgenerating search results comprising web documents with associatedexpert information.

BACKGROUND OF THE INVENTION

The World Wide Web (also referred to here as the “web”) provides a largecollection of interlinked information sources (in various formatsincluding texts, images, and media content) relating to virtually everysubject imaginable. As the web has grown, the ability of users to searchthis collection and identify content relevant to a particular subjecthas become increasingly important, and a number of search serviceproviders now exist to meet this need.

Conventional search services rely on indexing the content of various webpages. A user submits a search query comprising one or more searchterms; the search terms are matched against terms in an index of webcontent; and a list of results is generated based at least in part onhow well the content of particular pages matches the search terms.Simply matching terms, however, turns out not to be a reliable way ofproviding content relevant to the actual interest of a user.

More recently, efforts have been made to improve upon the conventionalsearch though a search system that allows for users who visit aparticular web page or site to evaluate it and make their evaluationspublic. User evaluations can be used to assist subsequent searchers. Forinstance, users may be able to “tag” the content item with keywords orlabels that describe the subject matter of the item; the tags assignedby various users can influence the system's response to subsequentqueries by that user or other users.

As users participate over time, however, search results may be inundatedwith user evaluations that are arbitrary or unrelated, resulting inoften unreliable and ineffective supplemented search results. Therefore,there exists a need for systems and methods for generating searchresults comprising web documents with reliable user evaluationinformation.

SUMMARY OF THE INVENTION

Generally, the present invention provides for systems, methods andcomputer program products for generating search results comprising webdocuments with associated expert information. The present invention isdirected toward a method for generating search results comprising webdocuments with associated expert information and includes receiving oneor more search queries, selecting one of the one or more search queries,determining one or more categories of web documents responsive to theselected search query and crawling a web graph of linked web documentsto identify one or more web documents tagged as within the one or morecategories responsive to the selected search query. The method furtherincludes generating a result set of the one or more web documentsidentified as within the one or more categories responsive to theselected search query, ranking the result set of the one or more webdocuments identified as within the one or more categories responsive tothe selected search query and generating a list of ranked search resultsresponsive to the selected search.

The present invention also provides for systems, methods and computerprogram products for supplementing a web graph of linked web documents.The method for supplementing a web graph of linked web documentsincludes identifying an existing web graph of linked web documents,identifying one or more experts for one of one or more categories,identifying one or more documents tagged by the one or more experts forthe given category and determining a corresponding category of the oneor more web documents tagged by the one or more experts for the givencategory. The method further includes assigning a proxy web document toa given one of the one or more experts identified for the given categoryand linking the one or more proxy web documents to the one or more webdocuments in the existing web graph tagged by the expert correspondingto the given category for which the expert and an expert for.

The present invention also provides for systems, methods and computerprogram products for determining one or more experts corresponding tocategories of web documents. The method includes identifying one or moreusers that input one or more tags, selecting one of the one or moreusers that input one or more tags, monitoring one or more secondaryusers' agreement with the one or more tags imputed by the selected userfor a given time period, and identifying the selected user as an expertfor one or more categories.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention is illustrated in the Figs. of the accompanying drawingswhich are meant to be exemplary and not limiting, in which likereferences are intended to refer to like or corresponding parts, and inwhich:

FIG. 1 illustrates a block diagram of a system for generating searchresults comprising web documents with associated expert informationaccording to one embodiment of the present invention;

FIG. 2 illustrates a flow diagram presenting a method for determiningone or more experts corresponding to categories of web documentsaccording to one embodiment of the present invention;

FIG. 3 illustrates a detailed flow diagram presenting a method fordetermining one or more experts corresponding to categories of webdocuments according to one embodiment of the present invention;

FIG. 4 illustrates a flow diagram presenting a method for supplementingthe web graph of linked documents according to one embodiment of thepresent invention;

FIG. 5 illustrates a detailed flow diagram presenting a method forsupplementing the web graph of linked documents according to oneembodiment of the present invention;

FIG. 6 illustrates a detailed flow diagram presenting a method forsupplementing the web graph of linked documents according to anotherembodiment of the present invention;

FIG. 7 illustrates a sample web graph of documents according to oneembodiment of the present invention;

FIG. 8 illustrates a flow diagram presenting a method for generatingsearch results comprising web documents with associated expertinformation according to one embodiment of the present invention;

FIG. 9 illustrates a flow diagram presenting a method for ranking searchresults comprising web documents with associated expert informationaccording to one embodiment of the present invention; and

FIG. 10 illustrates a detailed flow diagram presenting a method forranking search results comprising web documents with associated expertinformation according to one embodiment of the present invention.

DETAILED DESCRIPTION OF THE INVENTION

In the following description of the embodiments of the invention,reference is made to the accompanying drawings that form a part hereof,and in which is shown by way of illustration, exemplary embodiments inwhich the invention may be practiced. It is to be understood that otherembodiments may be utilized and structural changes may be made withoutdeparting from the scope of the present invention.

FIG. 1 illustrates one embodiment of a system for generating searchresults comprising web documents with associated expert information 100that includes a first client device 110, a second client device 120, acomputer network 130, a content provider 140, and a search provider 150.The search provider 150 comprises a central server 160, a searchdatabase 170, and a graph module 180. The central server 160 comprises auser interface 162, a search engine 164, a tag module 166 and anidentification module 168.

The computer network 130 may be any type of computerized network capableof transferring data, such as the Internet. According to one embodimentof the invention, the first client device 110 and the second clientdevice 120 are general purpose personal computers comprising aprocessor, transient and persistent storage devices, input/outputsubsystem and bus to provide a communications path between componentscomprising the general purpose personal computer. For example, a 3.5 GHzPentium 4 personal computer with 512 MB of RAM, 40 GB of hard drivestorage space and an Ethernet interface to a network. Other clientdevices are considered to fall within the scope of the present inventionincluding, but not limited to, hand held devices, set top terminals,mobile handsets, PDAs, etc. The present invention is not limited to onlythe first client device 110 and the second client device 120 and maycomprise additional, disparate client devices. The first client device110 and second client 120 are therefore presented for illustrativepurposes representative of multiple client devices.

According to one embodiment of the invention, the central server 160 isa programmable processor-based computer device that includes persistentand transient memory, as well as one or more network connection portsand associated hardware for transmitting and receiving data on thenetwork 130. The central server 160 may host websites, store data, serveads, etc. Those of skill in the art understand that any number and typeof central server 160, the first client device 110 and the second clientdevice 120 may be connected to the network 130.

The search engine 164, the tag module 166, the identification module 168and the graph module 180 may comprise one or more processing elementsoperative to perform processing operations in response to executableinstructions, collectively as a single element or as various processingmodules, which may be physically or logically disparate elements. Thesearch database 170 may be one or more data storage devices of anysuitable type, operative to store corresponding data therein. Those ofskill in the art recognize that the search provider 150 may utilize moreor fewer components and data stores, which may be local or remote withregard to a given component or data store.

In accordance with one embodiment, the first client device 110, thesecond client device 120, the search provider 150 and the contentprovider 140 are communicatively interconnected via the communicationsnetwork 130. The search provider 150 may maintain the search database170 which stores an index of one or more web documents accessed from thecontent provider 140 via communication pathways that the network 130provides. Examples of web documents include web pages, video clips,audio clips, etc.

In accordance with one embodiment, the first client device 110 or thesecond client device 120 may transmit one or more search requests thatcontain one or more search queries to the search engine 164 of thesearch provider 150 via the user interface 162. The search engine 164may handle the one or more incoming search requests and perform a searchof the index at the search database 170, as is known by those of skillin the art. The search engine 164 may then retrieve one or more relevantsearch results responsive to the search request from the search database170 and display the search results on the user interface 162. A user ofthe first client device 110 or the second client device 120 may view theone or more relevant search results.

A user of the first client device 110 or the second client device 120may have the option of tagging the one or more relevant search resultsas falling within a category topic. For example, a user of the firstclient 110 may review a search result for a web document, such as“www.espn.com” and tag the web document as falling within the category“Sports”. The tag module 166 may then receive and associate the one ormore tags for the one or web documents in the search database 170.Additionally, the tag module 166 may then transmit information regardingthe web document that has been tagged, the tag associated with the webdocument and the user who has tagged the web document to theidentification module 168. As used herein, the term “tag” refers to anyarbitrary item of metadata associated with a content item, for example,one or more terms or phrases associated with an item of information(such as text data, audio data, video data, image data, etc.). A giventag may describe a content item for classification purposes or fornon-classification purposes, and may comprise one or more folders,categories, etc.

In accordance with one embodiment, a user of the second client device120 transmits one or more search requests that contain one or moresearch queries to the search engine 164 of the search provider 150 viathe user interface 162. As described above, the search engine 164 mayhandle the one or more incoming search requests and perform a search ofthe index at the search database 170. The search engine 164 may identifyone or more relevant search results responsive to the search request anddisplay the search results on the user interface 162 with correspondingtags associated with a given a given search result. The user of thesecond client device 120 may have the option of agreeing with ordisagreeing with the tag associated with a given web documentcorresponding to a given search result.

The identification module 168 may receive information regardingagreement with the one or more tags associated with the one or webdocument. For example, a user of the second client 120 may review asearch result for a web document, such as “www.espn.com” with anassociated tag as falling within the category “Sports”, and submit thathe or she agrees with the association of the tag “Sports” with the webdocument. The identification module 168 may then record the number ofusers that agree with the one or more tags associated with the one ormore web documents for a given time period. The identification module168 may identify a user of the first client device 110 as an expert fora particular category based upon the number of users that agree with theone or more tags associated with the one or more web documents for agiven time period. Methods for determining one or more expertscorresponding to categories of web documents are described in greaterdetail below with respect to the description of FIGS. 2 through 3.

In accordance with one embodiment, the graph module 180 may organize theweb documents in a web graph format representative of the web documentsavailable in the search database 170, where web documents of the samecategory are linked together. The identification module 168 may transmitinformation regarding identification of experts to the graph module 180.The graph module 180 may in turn supplement the web graph of availableweb documents with information regarding categorization of the webdocuments by one or more experts, such information being stored in thesearch database 170. Methods for supplementing the web graph of linkeddocuments are described in greater detail below with respect to thedescription of FIGS. 4 through 6.

In accordance with one embodiment, a user of the first client device 110or the second client device 120 may subsequently transmit one or moresearch requests that contain one or more search queries to the searchengine 164 of the search provider 150 via the user interface 162. Asdescribed earlier, the search engine 164 may handle the one or moreincoming search requests and perform a search of the index at the searchdatabase 170. The search engine 164 may then retrieve one or morerelevant search results responsive to the search request from the searchdatabase 170 and display the search results on the user interface 162with corresponding tags associated with a given web documentcorresponding to a given search result and with corresponding expertinformation associated with a given web document. Methods for generatingsearch results comprising web documents with associated expertinformation will be described in further detail below with respect tothe description of FIGS. 8 through 12.

FIG. 2 illustrates a flow diagram presenting a method for determiningone or more experts corresponding to categories of web documentsaccording to one embodiment of the present invention. In accordance withthe embodiment of FIG. 2, the method comprises identifying one or moreusers that input one or more tags, step 210. One of the one or moreusers that input one or more tags may be selected, step 220. Forexample, a user viewing the web page located at the URL “www.espn.com”may tag one or more web pages as a “Sports” web documents. Agreement bysecondary or subsequent users with the one or more tags inputted by theselected user may be then monitored for a given time period, step 230.Continuing from the previous example, subsequent users who view thewebpage located at the URL “www.espn.com”, may also view the associatedtag “Sports” and have the option to agree or disagree with the “Sports”category tag. An identification module, such the identification module168 of FIG. 1, may monitor agreement or disagreement by subsequent userswith the given tag. The selected user may be then identified as expertfor one or more categories, step 240. For example, an identificationmodule may determine that, on the basis of the number of subsequent orsecondary users who agreed with the initial user who tagged the webpage“www.espn.com” with a “Sports” tag, is an expert for the tag category“Sports”.

FIG. 3 illustrates a detailed flow diagram presenting a method fordetermining one or more experts corresponding to categories of webdocuments according to one embodiment of the present invention. Inaccordance with the embodiment of FIG. 3, the method comprisesidentifying one or more users that input one or more tags, step 310,with a given one of the one or more users that input one or more tagsselected, step 320. Agreement by secondary users with the one or moretags inputted by the selected user may be monitored, step 330, and adetermination made as to whether the number of secondary users thatagree with the one or more tags inputted by the selected users exceeds athreshold value, step 340. Continuing from the previous example, adetermination may be made as to whether at least 1000 secondary usersagree with the tag “Sports” applied to the web page located at the URL“www.espn.com”, which was input by a first user.

If the number of secondary users that agree does not exceed a thresholdvalue, program flow may then return to step 320. If the number ofsecondary users that agree does exceed a threshold value, then the timeperiod in which secondary users agree with the one or more tags inputtedby the selected user may be monitored, step 350. A determination may bethen made as to whether the time period monitored exceeds a thresholdvalue, step 360. For example, if the determination made that 1000secondary users agreed with the tag “Sports” applied to the web pagelocated at the URL “www.espn.com” inputted by a first user, a seconddetermination may be made as to whether the 10000 secondary users agreedduring a 30 day time period. If the time period does not exceed athreshold value, program flow returns to step 320. If the time perioddoes exceeds a threshold value, then the selected user is identified asan expert for one or more categories, step 370.

FIG. 4 illustrates a flow diagram presenting a method for supplementingthe web graph of linked documents according to one embodiment of thepresent invention. In accordance with the embodiment of FIG. 4, themethod may begin by identifying an existing web graph of linkeddocuments, step 410, and identifying one or more experts for one of oneor more categories, step 420. One or more documents tagged by the one ormore experts for the given category may be then identified, step 430.The category of the one or more documents tagged by the one or moreexperts for the given category may be determined, step 440. Continuingfrom the previous example, the first user may be identified as an expertfor the category “Sports” based upon the subsequent analysis performedon the first user's tagging of the web page located at the URL“www.espn.com”. Other web documents tagged by the first user expert withthe tag “Sports” may then be identified.

A proxy document may be assigned to a given one of the one or moreexperts identified for the given category, step 450. One or more proxydocuments may be then linked to the one or more documents tagged by theexpert corresponding to the given category for which the expert is anexpert, step 460. For example, a proxy document representative of thefirst user expert may be implemented in a web graph representative of adatabase of web documents and linked to one or more web documents thefirst user expert has tagged with the “Sports” tag. To provide anadditional example, the first user expert may have tagged the webdocument located at the URL “www.foxsports.com” and the web documentlocated at the URL “www.sportsillustrated.cnn.com” with the tag“Sports”. The proxy document representative of the first user expert maybe linked to the three web documents in the web graph.

FIG. 5 illustrates a detailed flow diagram presenting a method forsupplementing the web graph of linked documents according to oneembodiment of the present invention. In accordance with the embodimentof FIG. 5, the method comprises identifying an existing web graph oflinked documents, step 510, and identifying one of the one or moreexperts for one or more categories, step 520. A given one of the one ormore categories for which the user is an expert for may be selected,step 530, and one or more documents tagged by the selected expert maythen be identified, step 540. The category of the one or more documentstagged by the selected expert may then be determined, step 550. Adetermination may be made as to whether the category of the one or moredocuments matches the category for which the selected user is an expert,step 560.

If the category of the one or more documents tagged does not match thecategory for which the selected expert is an expert for, program flowmay then return to step 530. Continuing from the previous example, ifthe first user expert tagged a web document with the tag “Cooking”, thedetermination would be made that the category of the tag for the webdocument does not match the category for which the first user is anexpert for, namely “Sports”. If the category of the one or moredocuments tagged matches the category for which the selected expert isan expert, a proxy document may be assigned to represent the expert,step 570. The proxy document may be then linked to the one or moredocuments tagged by the expert in the web graph, step 580.

FIG. 6 illustrates a detailed flow diagram presenting a method forsupplementing the web graph of linked documents according to anotherembodiment of the present invention. In accordance with the embodimentof FIG. 6, the method comprises identifying one or more experts for oneor more categories, step 610, and selecting one of the one or morecategories for which the expert is an expert, step 620. One or moredocuments tagged by the selected expert may be then identified, step630. The category of the one or more documents tagged by the selectedexpert is determined, step 640, and a determination may be made as towhether the category of the one or more documents matches the categoryfor which the selected expert is an expert, step 650.

If the category of the one or more documents tagged does not match thecategory for which the selected expert is an expert, program flow maythen return to step 620. If the category of the one or more documentstagged matches the category for which the selected expert is an expert,then the one or more documents tagged by the expert may be linked to theproxy document corresponding to the expert in the web graph, step 660.The one or more documents tagged by the expert may then be also linkedto the existing web documents that are linked to the proxy document inthe web graph.

FIG. 7 illustrates a block diagram presenting a sample web graph oflinked documents according to one embodiment of the present invention.Web documents 710,720 and 730 may be web documents identified in thesearch database maintained by a search provider that are linked togetheras they are web documents that fall within the category “Sports” orotherwise have a relationship, such as hyperlinks between the webdocuments 710, 720, 730. Web documents 740 and 750 are proxy webdocuments that represent users who have been identified as experts.According to FIG. 7, web document 740 represents Expert A who is anexpert for the category “Sports” and has tagged web document 710 and webdocument 720 with the tag “Sports”. Similarly, web document 750represents Expert B who is also an expert for the category “Sports” andhas tagged web document 720 with the tag “Sports”.

In another embodiment of the present invention, web document 710 mayrepresent a web document that was not stored within the index that thesearch provider maintains, but which was tagged by Expert A, representedby web document 740, as falling within the “Sports” category. Therefore,web document 710 may be implemented in the web graph and linked to webdocument 740, as well web documents 720 and 730, which also fall withinthe category of “Sports”.

FIG. 8 illustrates a flow diagram presenting a method for generatingsearch results comprising web documents with associated expertinformation according to one embodiment of the present invention. Inaccordance with the embodiment of FIG. 8, the method comprises receivingone or more search queries, step 810. A given one of the one of the oneor more search queries may be selected, step 820, and one or morecategories of documents responsive to the selected search query may bedetermined, step 830. According to one embodiment, the web graph iscrawled to identify one or more documents tagged as within the one ormore categories responsive to the selected search query, step 840, withthe generation of a result set of the one or more documents identifiedas within the one or more categories responsive to the selected searchquery, step 850. For example, a user may submit a search query for“sports scores” and a search engine maintained by a search provider maydetermine that documents that fall within the category of “Sports” wouldbe responsive to the search query.

For example, the engine may crawl the index at the search database thatis organized in a web graph format and return a result set comprisingthree web documents located at the following URLs: (i) www.espn.com,(ii) www.foxsports.com and (iii) www.sportsillustrated.cnn.com. Theresult set comprising one or more documents identified as within the oneor more categories responsive to the selected search query may beranked, step 860. For example, a ranking of the result set may be madeon the basis of the number of links that are linked to the web documentsor the number of experts that have tagged the web document. A list ofthe ranked search results responsive to the selected search query may begenerated, step 870. In another embodiment, the list of ranked searchresults responsive to the selected search query may be displayed inconjunction with the corresponding one or more experts that have taggeda given search result. In another embodiment, the list of ranked searchresults responsive to the selected search query may be displayed withthe corresponding web documents that are linked to a given searchresult.

FIG. 9 illustrates a flow diagram presenting a method for ranking searchresults comprising web documents with associated expert informationaccording to one embodiment of the present invention. In accordance withthe embodiment of FIG. 9, the method comprises generating a result setof one or more documents identified as within the one or more categoriesresponsive to the selected search query, step 910. The number ofdocuments that are linked to a given one of the documents in the resultset may then be determined, step 920. A determination may then be madeas to the number of experts that are linked to a given one of thedocuments in the result set, step 930.

Secondary characteristics for a given one of the documents in the resultthen may then be determined, step 940. The number of linked documentsand the number of experts, as well as secondary characteristics for agiven one of the documents in the result set, may be compared, step 950.The result set of the one or more documents identified as falling withinthe one or more categories responsive to the selected search query maybe ranked on the basis of the relative number of linked documents, thenumber of experts and secondary characteristics for a given one of thedocuments in the result set, step 960. A list of ranked search resultsresponsive to the selected search query may then be generated, step 970.

FIG. 10 illustrates a detailed flow diagram presenting a method forranking search results comprising web documents with associated expertinformation according to one embodiment of the present invention. Inaccordance with the embodiment of FIG. 10, the method comprisesgenerating a result set of the one or more documents identified aswithin the one or more categories responsive to the selected searchquery, step 1010. The number of documents that are linked to a given oneof the documents in the result set may then be determined, step 1020,and one of the one or more documents identified as within the one ormore categories responsive to the selected search query in the resultset may be selected, step 1030.

A second, or next one of the one or more documents identified as withinthe one or more categories responsive to the selected search query inthe result set may be selected, step 1040. The number of documents thatare linked to the first document identified in the result set may thenbe compared to the number of documents that are linked to the nextdocument identified in the result set, step 1050. Similarly, the numberof experts that are linked to the first document identified in theresult set may be compared with the number of experts that are linked tothe next document identified in the result set. Alternatively, or inconjunction with the foregoing, one or more secondary characteristicsfor the first document identified in the result set may be compared tothe one or more secondary characteristics for the next documentidentified in the result set.

A determination may then be made as to whether the number of documentslinked to the next document is greater than the number of documentslinked to the first document, step 1060. If the number of documentslinked to the next document is greater than the number of documentslinked to the first document, than the next document may be rankedhigher than the first document, step 1070. Otherwise, the next documentmay be ranked lower than the first document, step 1080. Program flow maythen return to step 1040, where a third or next one of the one or moredocuments identified as within the one or more categories responsive tothe selected search query in the result set is selected. If there remainno documents identified as within the one or more categories responsiveto the selected search query in the result set, the method will end,step 1090.

According to another embodiment, which may be performed in conjunctionwith other embodiments described herein, a determination may be made asto whether the number of experts linked to the next document is greaterthan the number of documents linked to the first document, step 1160. Ifthe number of experts linked to the next document is greater than thenumber of experts linked to the first document, than the next documentmay be ranked higher than the first document, step 1170. Otherwise, thenext document may be ranked lower than the first document, step 1180.According to still further embodiments of the present invention, whichmay be performed in conjunction with other embodiments described herein,a determination may be made as to whether the quality of the one or moresecondary characteristics for the next document is greater than thequality of the one or more secondary characteristics for the firstdocument, step 1260. If the quality of the one or more secondarycharacteristics for the next document is greater than quality of the oneor more secondary characteristics for the first document, than the nextdocument may be ranked higher than the first document, step 1270.Otherwise, the next document may be ranked lower than the firstdocument, step 1280.

Continuing from the previous example, the result set may comprise threeweb documents located at the following URLs: (i) www.espn.com, (ii)www.foxsports.com and (iii) www.sportsillustrated.cnn.com which areresponsive to the search query “Sports scores” as the three webdocuments fall within the “Sports” category. The web document located atthe URL www.espn.com may have five web documents linked to it within aweb graph, the web document located at the URL www.foxsports.com mayhave three web documents linked to it, and the web document located atthe URL www.sportsillustrated.cnn.com may have one web document linkedto it. Therefore, the result set of web documents would be ranked withwww.espn.com listed first, www.foxsports.com listed second andwww.sportsillustrated.cnn.com listed third. Similarly, the web documentlocated at the URL “www.espn.com” may have been tagged by four expertsas falling with the category “Sports”, the web document located at theURL “www.foxsports.com” may have been tagged by two experts, and the webdocument located at the URL “www.sportsillustrated.cnn.com” may have notbeen tagged by an expert. Therefore, the result set of web documents maybe ranked with “www.espn.com” listed first, “www.foxsports.com” listedsecond and “www.sportsillustrated.cnn.com” listed third.

Therefore, the present invention provides for systems and methods forgenerating search results comprising web documents with reliable userevaluation information by creating expert user evaluations andassociating such evaluations with corresponding search results.

FIGS. 1 through 10 are conceptual illustrations allowing for anexplanation of the present invention. It should be understood thatvarious aspects of the embodiments of the present invention could beimplemented in hardware, firmware, software, or combinations thereof. Insuch embodiments, the various components and/or steps would beimplemented in hardware, firmware, and/or software to perform thefunctions of the present invention. That is, the same piece of hardware,firmware, or module of software could perform one or more of theillustrated blocks (e.g., components or steps).

In software implementations, computer software (e.g., programs or otherinstructions) and/or data is stored on a machine readable medium as partof a computer program product, and is loaded into a computer system orother device or machine via a removable storage drive, hard drive, orcommunications interface. Computer programs (also called computercontrol logic or computer readable program code) are stored in a mainand/or secondary memory, and executed by one or more processors(controllers, or the like) to cause the one or more processors toperform the functions of the invention as described herein. In thisdocument, the terms “machine readable medium,” “computer program medium”and “computer usable medium” are used to generally refer to media suchas a random access memory (RAM); a read only memory (ROM); a removablestorage unit (e.g., a magnetic or optical disc, flash memory device, orthe like); a hard disk; electronic, electromagnetic, optical,acoustical, or other form of propagated signals (e.g., carrier waves,infrared signals, digital signals, etc.); or the like.

Notably, the Figs. and examples above are not meant to limit the scopeof the present invention to a single embodiment, as other embodimentsare possible by way of interchange of some or all of the described orillustrated elements. Moreover, where certain elements of the presentinvention can be partially or fully implemented using known components,only those portions of such known components that are necessary for anunderstanding of the present invention are described, and detaileddescriptions of other portions of such known components are omitted soas not to obscure the invention. In the present specification, anembodiment showing a singular component should not necessarily belimited to other embodiments including a plurality of the samecomponent, and vice-versa, unless explicitly stated otherwise herein.Moreover, applicants do not intend for any term in the specification orclaims to be ascribed an uncommon or special meaning unless explicitlyset forth as such. Further, the present invention encompasses presentand future known equivalents to the known components referred to hereinby way of illustration.

The foregoing description of the specific embodiments will so fullyreveal the general nature of the invention that others can, by applyingknowledge within the skill of the relevant art(s) (including thecontents of the documents cited and incorporated by reference herein),readily modify and/or adapt for various applications such specificembodiments, without undue experimentation, without departing from thegeneral concept of the present invention. Such adaptations andmodifications are therefore intended to be within the meaning and rangeof equivalents of the disclosed embodiments, based on the teaching andguidance presented herein. It is to be understood that the phraseologyor terminology herein is for the purpose of description and not oflimitation, such that the terminology or phraseology of the presentspecification is to be interpreted by the skilled artisan in light ofthe teachings and guidance presented herein, in combination with theknowledge of one skilled in the relevant art(s).

While various embodiments of the present invention have been describedabove, it should be understood that they have been presented by way ofexample, and not limitation. It would be apparent to one skilled in therelevant art(s) that various changes in form and detail could be madetherein without departing from the spirit and scope of the invention.Thus, the present invention should not be limited by any of theabove-described exemplary embodiments, but should be defined only inaccordance with the following claims and their equivalents.

What is claimed is:
 1. A method for ranking search results comprising: receiving from a client device at least one search query; identifying with a provider device coupled to the client device over a computer network search results responsive to the query; and communicating the search results to the client device, the search results comprising a link to each of a plurality of documents, the plurality of documents ranked therein based at least on a number of experts linked to each of the plurality of the documents.
 2. The method of claim 1, comprising identifying at least one category of documents in the search results, and wherein the number of experts linked to each of the plurality of documents comprise a number of experts that have tagged each of the plurality of documents as belonging to the at least one category.
 3. The method of claim 2, wherein the provider device deems a user an expert by receiving a tag specified by the user for each of a plurality of documents in at least one search result set, the user indicating therewith that the tagged document belongs to at least one category of documents and communicating the tagged documents to a plurality of other users, wherein the provider device deems the user an expert based on a number of other users that agree with the tags specified by the user.
 4. The method of claim 3, wherein the provider device deems the user an expert based on a number of other users that agree with the tags specified by the user within a given period of time.
 5. The method of claim 3, wherein the provider device deems the user an expert based on a number of other users that agree with the tags specified by the user exceeding a threshold value.
 6. The method of claim 3, wherein the provider device deems the user an expert based on a number of other users that agree with the tags specified by the user within a given period of time exceeding a threshold value.
 7. A system for ranking search results comprising at least one provider device coupled over a computer network to at least one client device, the provider device operable at least to: receive from the client device at least one search query; identify search results responsive to the query; and communicate the search results to the client device, the search results comprising a link to each of a plurality of documents, the plurality of documents ranked therein based at least on a number of experts linked to each of the plurality of the documents.
 8. The system of claim 7, the provider device further operable to identify at least one category of documents in the search results, and wherein the number of experts linked to each of the plurality of documents comprise a number of experts that have tagged each of the plurality of documents as belonging to the at least one category.
 9. The system of claim 8, the provider device further operable receive a tag specified by the user for each of a plurality of documents in at least one search result set, the user indicating therewith that the tagged document belongs to at least one category of documents and communicate the tagged documents to a plurality of other users, wherein the provider device deems the user an expert based on a number of other users that agree with the tags specified by the user.
 10. The system of claim 8, wherein the provider device deems the user an expert based on a number of other users that agree with the tags specified by the user within a given period of time.
 11. The system of claim 8, wherein the provider device deems the user an expert based on a number of other users that agree with the tags specified by the user exceeding a threshold value.
 12. The system of claim 8, wherein the provider device deems the user an expert based on a number of other users that agree with the tags specified by the user within a given period of time exceeding a threshold value.
 13. A non-transitory computer readable medium having program code stored thereon that when executed causes a provider device to perform a method for ranking search results comprising: receiving from a client device at least one search query; identifying with the provider device coupled to the client device over a computer network search results responsive to the query; and communicating the search results to the client device, the search results comprising a link to each of a plurality of documents, the plurality of documents ranked therein based at least on a number of experts linked to each of the plurality of the documents.
 14. The computer readable medium of claim 13, the method further comprising identifying at least one category of documents in the search results, and wherein the number of experts linked to each of the plurality of documents comprise a number of experts that have tagged each of the plurality of documents as belonging to the at least one category.
 15. The computer readable medium of claim 14, wherein the provider device deems a user an expert by receiving a tag specified by the user for each of a plurality of documents in at least one search result set, the user indicating therewith that the tagged document belongs to at least one category of documents and communicating the tagged documents to a plurality of other users, wherein the provider device deems the user an expert based on a number of other users that agree with the tags specified by the user.
 16. The computer readable medium of claim 14, wherein the provider device deems the user an expert based on a number of other users that agree with the tags specified by the user within a given period of time.
 17. The computer readable medium of claim 14, wherein the provider device deems the user an expert based on a number of other users that agree with the tags specified by the user exceeding a threshold value.
 18. The computer readable medium of claim 14, wherein the provider device deems the user an expert based on a number of other users that agree with the tags specified by the user within a given period of time exceeding a threshold value. 